Improved Simulation of Image Detail Visibility using the Non-Subsampled Contourlet Transform

A previous study proposed a method for simulation of detail visibility of natural images (from different observation distances) by using contrast sensitivity functions and wavelets (Pedersen and Farup, Color and Imaging Conference 2012). In this paper we propose an improved method using the non-subsampled contourlet transform, which accounts for important aspects of the human visual system such as orientation sensitivity. In addition we account for the effect of surround illumination. Objective and subjective evaluations show that the proposed methodology is promising, and that it introduces fewer artifacts than the previous method. Introduction Pedersen and Farup [1] presented at the 20th Color and Imaging Conference a method for simulating image detail visibility of natural images, where perceptible information at a given distance is kept, while imperceptible information is discarded. If the original and the simulated image are viewed from the simulated distance or farther away, they should be indistinguishable. This is because the information from the original, that is imperceptible, has been removed through the simulation. Such a method to simulate detail visibility has several applications; it can be used in the evaluation of image quality, improve compression, gamut mapping, and halftoning. The method is based on the relationship between contrast sensitivity and spatial frequency (contrast sensitivity functions), and an octave-wise spread over the spatial frequency range matched by wavelet decompositions. The method showed promising results; giving a goodmatch to human observers and improving the performance of an image quality metric. The method had several advantageous properties, such as multi-resolution from coarse to fine resolutions and the basis elements in the representation are localized in both spatial and the frequency domains. However, since it uses wavelets it suffers from artifacts, such as blocking and ringing. These artifacts are unwanted, and if they are suprathreshold they become visible for observers. Wavelets have a crude directional representation (primarily vertical, horizontal, and diagonal). Although they are good at representing point discontinuities, they are not good at representing discontinuities along edges. Therefore, the representation should contain basis elements oriented in a variety of directions, preferably in more directions than those offered bywavelets. Additionally, wavelets are isotropic, and they are therefore not able to to capture smooth contours in images, which often occurs in natural images. Hence, the representation should contain basis elements using a variety of elongated shapes with different aspect ratios, which can be accomplished with for example curvelets [2] or contourlets [3]. Furthermore, the model by Pedersen and Farup [1] does not incorporate a model of the Contrast Sensitivity Function (CSF) that accounts for the fact that we are less sensitive oblique angles than vertical and horizontal angles [4, 5]. In this work we extend the method proposed by Pedersen and Farup [1] by increasing the directionality and better capture smooth contours through the use of the NonSubsampled Contourlet Transform (NSCT) [6]. Additionally, we incorporate a more advanced CSF model from Barten [7]. All with the goal of improving the simulation of image detail visibility of natural images. This paper is organized as follows: first we introduce relevant background, then we present the proposed methodology. Further, evaluation of the proposed method is presented. At last we conclude and propose future work. Background The method proposed by Pedersen and Farup [1] was based on several different works [8–11]. The image to be filtered was first converted into a suitable color space. For this a linear RGB color space inspired by the YCbCr color space was used, where the primaries were defined according to the wavelengths of the monochromatic light sources used to generate the gratings used in the experiments by Mullen [12] to measure the chromatic CSFs. Further, the filtering method was based on existing work on local bandlimited contrast for complex images [8] and wavelet based contrast sensitivity filtering [9]. A wavelet decomposition of the image was carried out to obtain octave width bands of frequencies. The coefficients were reconstructed to fullscale before CSFs are applied. The CSF filtered coefficients were compared to the contrast low-pass band, if the filtered coefficients were higher than the low-pass coefficients, the information was kept, else it was removed. Additionally, contrast masking was incorporated using an extended intra channel masking method accounting for local activity [13]. For more information we refer the reader to Pedersen and Farup [1]. The non-subsampled contourlet transform The NSCT was proposed by da Cunha et al. [6], and it was based on a non-subsampled pyramid structure and non-subsampled directional filter banks. The design results in a flexible multi-scale, multi-direction, and shift in21st Color and Imaging Conference Final Program and Proceedings 191 variant image decomposition. The NSCT can be divided into two shift-invariant parts: a non-subsampled pyramid structure to achieve multi-scale properties and a nonsubsampled filter bankwith directional filters. The first part gave a sub-band decomposition similar to that of the Lapacian pyramid, which was done through the usage of twochannel non-subsampled 2-D filter banks. For the latter, a directional filter was constructed by combining criticallysampled two-channel fan filter banks and re-sampling operations. This results in a tree-structured filter bank that splits the 2-D frequency plane into directional wedges. These two elements were combined as seen in Figure 1. The NSCT has shown to be effective for image quality assessment [14], image fusion [15], face recognition [16], denoising [6, 17, 18], and enhancement [6, 19]. It is therefore likely, together with its advantageous properties, that it will be effective also for the simulation of image detail visibility. Figure 1. Non-Subsampled Contourlet Transform (NSCT). On the left the nonsubsampled filter bank structure and on the right the idealized frequency partitioning obtained by the non-subsampled filter bank. Figure reproduced from Lu et al. [14]. Proposed methodology The input image is transformed into theYbr color space as proposed by Pedersen and Farup [1]. Then each channel is decomposed using the NSCT. Unless stated otherwise we decompose the image using three levels with 4, 8, and 16 orientations, where the pyramidal filter generated from a 1-D filter using a maximally flat mapping function with 4 vanishing moments, and the directional filter is a 2-D diamond maxflat filter of order 7. These are the default filters proposed by da Cunha et al. [6]. This gives us a low-pass filtered version (LL) and high-pass filtered versions hψ in several orientations ψ. The high-pass filtered coefficients are filtered with CSFs. For the achromatic channel (Y) a luminance CSF is applied, and for the two chromatic channels (b and r) chromatic CSFs are applied. Due to the division into orientations using the non-subsampled contourlet, the CSF needs to be adapted to the orientation. We apply the CSF model from Barten [7] that incorporate orientation dependence of the CSF and the effect of surround illumination. The general formula for the luminance CSF is